3D keypoint detectors and descriptors for 3D objects recognition with TOF camera

International audienceThe goal of this work is to evaluate 3D keypoints detectors and descriptors, which could be used for quasi real time 3D object recognition. The work presented has three main objectives: extracting descriptors from real depth images, obtaining an accurate degree of invariance and robustness to scale and viewpoints, and maintaining the computation time as low as possible. Using a 3D time-of-flight (ToF) depth camera, we record a sequence for several objects at 3 different distances and from 5 viewpoints. 3D salient points are then extracted using 2 different curvatures-based detectors. For each point, two local surface descriptors are computed by combining the shape index histogram and the normalized histogram of angles between the normal of reference feature point and the normals of its neighbours. A comparison of the two detectors and descriptors was conducted on 4 different objects. Experimentations show that both detectors and descriptors are rather invariant to variations of scale and viewpoint. We also find that the new 3D keypoints detector proposed by us is more stable than a previously proposed Shape Index based detector

Détecteurs de points d'intérêt 3D basés sur la courbure

National audienceDans cet article, nous proposons un nouveau détecteur de points d'intérêt 3D (keypoint). Notre sélection des points saillants se base sur l'expression de la variation locale de la surface à travers les courbures principales calculées sur un nuage de points ordonnés, associé à une seule vue (deux dimensions et demie). Nous avons comparé sept méthodes qui combinent ces courbures et extraient des keypoints en se basant sur: 1) un seuillage des valeurs d'un facteur de qualité: Quality Factor (FQ), 2) un seuillage sur une mesure de l'indice de forme: Shape Index (SI), 3) les composantes connexes d'une carte de classification basée sur SI, 4) les composantes connexes d'une carte de classification basée sur SI et l'intensité de courbure : Curvedness (C), 5) les composantes connexes d'une carte de classification basée sur la courbure gaussienne (H) et la courbure moyenne (K), 6) une combinaison des deux derniers critères 4 et 5 (SC_HK) avec un tri final selon C et 7) une combinaison des trois critères 1, 4 et 5 (SC_HK_FQ). Une évaluation de la performance de ces détecteurs en termes de stabilité et répétabilité, montre la supériorité des deux nouveaux détecteurs SC_HK et SC_HK_FQ

Fast 3D keypoints detector and descriptor for view-based 3D objects recognition

International audienceIn this paper, we propose a new 3D object recognition method that employs a set of 3D keypoints extracted from point cloud representation of 3D views. The method makes use of the 2D organization of range data produced by 3D sensor. Our novel 3D interest points approach relies on surface type classifi-cation and combines the Shape Index (SI) - curvedness(C) map with the Gaus-sian (H) - Mean (K) map. For each extracted keypoint, a local description using the point and its neighbors is computed by joining the Shape Index histogram and the normalized histogram of angles between normals. This new proposed descriptor IndSHOT stems from the descriptor CSHOT (Color Signature of Histograms of OrienTations) which is based on the definition of a local, robust and invariant Reference Frame RF. This surface patch descriptor is used to find the correspondences between query-model view pairs in effective and robust way. Experimental results on Kinect based datasets are presented to validate the proposed approach in view based 3D object recognition

3D Object Recognition and Facial Identification Using Time-averaged Single-views from Time-of-flight 3D Depth-Camera

International audienceWe report here on feasibility evaluation experiments for 3D object recognition and person facial identification from single-view on real depth images acquired with an “off-the-shelf” 3D time-of-flight depth camera. Our methodology is the following: for each person or object, we perform 2 independent recordings, one used for learning and the other one for test purposes. For each recorded frame, a 3D-mesh is computed by simple triangulation from the filtered depth image. The feature we use for recognition is the normalized histogram of directions of normal vectors to the 3D-mesh facets. We consider each training frame as a separate example, and the training is done with a multilayer perceptron with 1 hidden layer. For our 3D person facial identification experiments, 3 different persons were used, and we obtain a global correct rank-1 recognition rate of up to 80%, measured on test frames from an independent 3D video. For our 3D object recognition experiment, we have considered 3 different objects, and obtain a correct single-frame recognition rate of 95%, and checked that the method is quite robust to variation of distance from depth camera to object. These first experiments show that 3D object recognition or 3D face identification, with a time-of-flight 3D camera, seems feasible, despite the high level of noise in the obtained real depth images

3D object recognition with points of interest

Soutenue par les progrès récents et rapides des techniques d'acquisition 3D, la reconnaissance d'objets 3D a suscité de nombreux efforts de recherche durant ces dernières années. Cependant, il reste à résoudre dans ce domaine plusieurs problématiques liées à la grande quantité d'information, à l'invariance à l'échelle et à l'angle de vue, aux occlusions et à la robustesse au bruit.Dans ce contexte, notre objectif est de reconnaitre un objet 3D isolé donné dans une vue requête, à partir d'une base d'apprentissage contenant quelques vues de cet objet. Notre idée est de formuler une méthodologie locale qui combine des aspects d'approches existantes et apporte une amélioration sur la performance de la reconnaissance.Nous avons opté pour une méthode par points d'intérêt (PIs) fondée sur des mesures de la variation locale de la forme. Notre sélection de points saillants est basée sur la combinaison de deux espaces de classification de surfaces : l'espace SC (indice de forme- intensité de courbure), et l'espace HK (courbure moyenne-courbure gaussienne).Dans la phase de description de l'ensemble des points extraits, nous proposons une signature d'histogrammes, qui joint une information sur la relation entre la normale du point référence et les normales des points voisins, avec une information sur les valeurs de l'indice de forme de ce voisinage. Les expérimentations menées ont permis d'évaluer quantitativement la stabilité et la robustesse de ces nouveaux détecteurs et descripteurs.Finalement nous évaluons, sur plusieurs bases publiques d'objets 3D, le taux de reconnaissance atteint par notre méthode, qui montre des performances supérieures aux techniques existantes.There has been strong research interest in 3D object recognition over the last decade, due to the promising reliability of the 3D acquisition techniques. 3D recognition, however, conveys several issues related to the amount of information, to scales and viewpoints variation, to occlusions and to noise.In this context, our objective is to recognize an isolated object given in a request view, from a training database containing some views of this object. Our idea is to propose a local method that combines some existent approaches in order to improve recognition performance.We opted for an interest points (IPs) method based on local shape variation measures. Our selection of salient points is done by the combination of two surface classification spaces: the SC space (Shape Index-Curvedness), and the HK space (Mean curvature- Gaussian curvature).In description phase of the extracted set of points, we propose a histogram based signature, in which we join information about the relationship between the reference point normal and normals of its neighbors, with information about the shape index values of this neighborhood. Performed experiments allowed us to evaluate quantitatively the stability and the robustness of the new proposed detectors and descriptors.Finally we evaluate, on several public 3D objects databases, the recognition rate attained by our method, which outperforms existing techniques on same databases

Reconnaissance d’objets 3D par points d’intérêt

There has been strong research interest in 3D object recognition over the last decade, due to the promising reliability of the 3D acquisition techniques. 3D recognition, however, conveys several issues related to the amount of information, to scales and viewpoints variation, to occlusions and to noise.In this context, our objective is to recognize an isolated object given in a request view, from a training database containing some views of this object. Our idea is to propose a local method that combines some existent approaches in order to improve recognition performance.We opted for an interest points (IPs) method based on local shape variation measures. Our selection of salient points is done by the combination of two surface classification spaces: the SC space (Shape Index-Curvedness), and the HK space (Mean curvature- Gaussian curvature).In description phase of the extracted set of points, we propose a histogram based signature, in which we join information about the relationship between the reference point normal and normals of its neighbors, with information about the shape index values of this neighborhood. Performed experiments allowed us to evaluate quantitatively the stability and the robustness of the new proposed detectors and descriptors.Finally we evaluate, on several public 3D objects databases, the recognition rate attained by our method, which outperforms existing techniques on same databases.Soutenue par les progrès récents et rapides des techniques d'acquisition 3D, la reconnaissance d'objets 3D a suscité de nombreux efforts de recherche durant ces dernières années. Cependant, il reste à résoudre dans ce domaine plusieurs problématiques liées à la grande quantité d'information, à l'invariance à l'échelle et à l'angle de vue, aux occlusions et à la robustesse au bruit.Dans ce contexte, notre objectif est de reconnaitre un objet 3D isolé donné dans une vue requête, à partir d'une base d'apprentissage contenant quelques vues de cet objet. Notre idée est de formuler une méthodologie locale qui combine des aspects d'approches existantes et apporte une amélioration sur la performance de la reconnaissance.Nous avons opté pour une méthode par points d'intérêt (PIs) fondée sur des mesures de la variation locale de la forme. Notre sélection de points saillants est basée sur la combinaison de deux espaces de classification de surfaces : l'espace SC (indice de forme- intensité de courbure), et l'espace HK (courbure moyenne-courbure gaussienne).Dans la phase de description de l'ensemble des points extraits, nous proposons une signature d'histogrammes, qui joint une information sur la relation entre la normale du point référence et les normales des points voisins, avec une information sur les valeurs de l'indice de forme de ce voisinage. Les expérimentations menées ont permis d'évaluer quantitativement la stabilité et la robustesse de ces nouveaux détecteurs et descripteurs.Finalement nous évaluons, sur plusieurs bases publiques d'objets 3D, le taux de reconnaissance atteint par notre méthode, qui montre des performances supérieures aux techniques existantes

3D Keypoints Detection for Objects Recognition

Abstract- In this paper, we propose a new 3D object recognition method that employs a set of 3D local features extracted from point cloud representation of 3D views. The method makes use of the 2D organization of range data produced by 3D sensor. A detector of 3D interest points requires the expression of the local surface variation around points. In our case, we opted for a curvature-based approach. We test six methods which combine principles curvatures values under the form of: 1) a measure of the Shape Index (SI), 2) a measure of a Quality Factor (FQ), 3) a map of Shape Index (SI) and curvedness(C), 4) a map of Gaussian (H) and Mean (K) curvatures, 5) a combination of 3 and 4 (SC_HK) and 6) a combination of 5 and 4(SC_HK_FQ). For each extracted point, a local description using the point and its neighbors is done by combining the shape index histogram and the normalized histogram of angles between normals. This local surface patch representation is used to find the correspondences between a model-test view pair. Performance evaluation of the detectors in terms of stability and repeatability shows the robustness of the proposed detectors to viewpoint variations. Experimental results on the Minolta data set are presented to demonstrate the efficiency of the proposed approach in view based object recognition

HMM-based gait modeling and recognition under different walking scenarios

International audienceThis paper addresses gait recognition, the problem of identifying people by the way of their walk. The proposed system consists of a model-free approach which extracts features directly from the human silhouette. The dynamics of the gait are modeled using Hidden Markov Models. Experiments have been carried out on the CASIA dataset C consisting of 153 people under four walking scenarios: normal walking, slow walking, fast walking and walking while carrying a bag. The results obtained are promising and compare favorably with existing approache